Face recognition has numerous security-related applications such as user identity matching, user authentication for online and offline resource accesses, door and gate unlocking, and person identification.
Conventionally, face recognition methods use a single captured image of the subject's face to find a match in a library of previously captured facial images. The face matching process is relatively accurate in comparison to other biometric identification methods with well-developed and tested facial feature analysis techniques. In general, the techniques extract feature data of a face region, which is unique for each person, from the captured image of the subject's face, and compares the feature data against those stored in the library, thereby matching the subject's face to that of a previously registered person.
U.S. Pat. No. 6,922,478 discloses one such conventional method for verifying the authenticity of an image recorded in a person identifying process and belonging to a person to be identified. However, one shortcoming of this method is that it maybe deceived by presenting to the system a photograph of another person's face for matching (photo spoofing). In this case, identity theft may not be prevented by two-dimensional (2D) based face recognition systems. FIG. 1A illustrates an exemplary conventional 2D-based face identification and authentication method. In this example, two persons appeared before the camera of a conventional face recognition system, which can be a kiosk or a personal computer. The faces of the two persons (101 and 102) were image-captured and recognized. FIG. 1B illustrates an exemplary photo spoofing being attempted on the conventional face recognition system. In this example, two persons appeared before the camera of the same face recognition system. Each person was holding a photograph (103 or 104) of the face of another person over her own face for being image-captured by the conventional face recognition system. The face recognition system will not be able to distinguish a person's actual face from a photograph of a face.
Improvements have been made with three-dimensional (3D) based face recognition. For example, U.S. Pat. No. 7,436,988 disclosed a method of 3D face recognition, taking a 3D triangular facial mesh as input and extracting the bilateral symmetry plane of the face surface for further analysis. However, even with a 3D based face recognition system, the system may still be deceived by a facemask or a 3D facial sculpture (facemask spoofing). Further, such systems usually require expensive dedicated equipment, such as the 3D scanners as described in U.S. Pat. No. 7,436,988, inhibiting their wide adoption.
Other face recognition systems have been developed to tackle photo spoofing and facemask spoofing by ways of detecting whether the subject is a living person, including requiring the subject to make multiple facial expressions and movements during an identification session and allowing the system to detect and capture the multiple frames of facial expressions and movements for matching. More sophisticated perpetrators of identity theft, however, have devised circumventive measures using series of photographs, electronic displays showing several views of the target's faces, and facemasks capable of projecting different facial expressions.
European Patent No. 1990770 discloses a face authentication apparatus implemented as a user terminal having a built-in camera. Its authentication process includes requiring the user to input a pattern and to make certain eye movements for living body detection. However, the apparatus would necessary employs a rather large display screen in order to place the direction pointers in disparate locations on the display screen for prompting sufficiently large and detectable eye movements. This inhibits its wide adoption, and is likely restricting its implementation to customized user terminals only.
U.S. Patent Application Publication No. 2014/0376787A1 discloses a personal authentication apparatus comprises a face detection unit configured to detect a face region of a person; a facial expression determination unit configured to determine a facial expression from the face region; a storage unit configured to store feature data used to authenticate a person in correspondence with respective facial expressions of a plurality of faces; and a selection unit configured to select feature data corresponding to the facial expression determined by the facial expression determination unit from the storage unit. This personal authentication apparatus employs facial expression analysis in its authentication process, but its design and purpose is to enhance the precision in recognizing and identifying a person's face. It is used to specifically tackle the problem of that the difference between the facial expression of the subject and that upon registration causes precision deterioration of personal authentication. In this case, this personal authentication apparatus would not be able to detect whether the subject is a living person.